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ABSTRACT 



Transforming the size of an input position indicator, such as 
a cursor, within a window presented by a display screen of 
a computer system. The window typically includes borders 
defining an enclosed presentation area for displaying infor- 
mation on the display screen. In response to determining the 
position of the input position indicator within the window, 
the size of the input position indicator can be transformed 
based upon that position determination. 

10 Claims, 22 Drawing Sheets 
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SYSTEM AND METHOD FOR RESIZING AN 
INPUT POSITION INDICATOR FOR A USER 
INTERFACE OF A COMPUTER SYSTEM 

STATEMENT REGARDING RELATED 5 
APPLICATIONS 

This is a divisional of U.S. patent application Ser. No. 
08/649,828, now U.S. Pat. No. 5,880,773, entitled "Display 
System and Method for Displaying Windows of an Operat- 
ing System to Provide a Three-Dimensional Workspace for 
a Computer System," filed Apr. 30, 1996, and currently 
pending. 

FIELD OF THE INVENTION 

15 

The present invention relates generally to a display sys- 
tem for managing and manipulating information on a com- 
puter display screen operating in a window display 
environment, and more particularly, relates to a display 
system for managing and manipulating the displayed win- 20 
dows of the operating system to provide a virtual three- 
dimensional workspace for the computer system. 

BACKGROUND OF THE INVENTION 

In a physical environment, such as on a desk or table, an 25 
individual may utilize certain areas for placement of docu- 
ments for convenient viewing. The surface area on the desk 
used to support the documents for viewing may be termed a 
workspace. Because the physical space on a desk for work- 
ing is limited multiple documents may be spread out or 30 
arranged in some systematic manner within the workspace. 
When multiple documents require more surface area than 
the available area of the workspace, documents may be 
stacked, within the workspace, with identifying portions 
being visible. 35 

In a computer system, the workspace for viewing docu- 
ments or applications is the display screen or monitor of the 
computer system. The operating system of the computer 
system generally manages or controls the documents or ^ 
applications displayed on the monitor. Early computer sys- 
tems displayed only one document or application on the 
display screen at a time, therefore limiting the effective 
computer workspace. In order to switch among multiple 
applications or documents, the user had to close the appli- 45 
cation or document being displayed and open a different 
application or document. Such a computer system was 
cumbersome. With the introduction of multi-tasking and 
graphical interfaces, the display screen was divided into 
multiple portions generally referred to as windows. A win- SQ 
dow is a portion of the display screen that may contain its 
own document or message. 

In window-based programs, the screen can be divided into 
several windows, each of which has its own boundaries and 
can contain a different document. The operating system 55 
controls these specially delineated window areas of the 
screen. Each window can act independently, as if it were a 
virtual display device. In window display environments, the 
windows may be resized and moved around on the display 
screen. Thus, with the advent of window-based programs, go 
the computer screen workspace became more flexible. The 
ability to resize and move windows as well as to overlay or 
stack windows on top of each other essentially provided the 
same type of workspace available on a physical two- 
dimensional tabletop workspace except on a smaller scale. 55 

Present day computer systems often employ "object- 
oriented" displays with windows, icons, pictures, text, pop- 
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up or drop-down menus, dialog boxes, and other graphical 
items or objects in what is known as a "Graphical User 
Interface" ("GUP'). In such systems, various combinations 
of graphical items or objects are utilized to convey infor- 
mation to users via the monitor or display. Users interact 
with the computer system by activating graphical items or 
objects on the display. The graphical items or objects are 
often activated by pointing at them with a cursor controlled 
by a mouse or pen and clicking a control button on the 
mouse or pen. 

An example of a computer graphical user interface that 
uses icons and windows is the Microsoft "WINDOWS"™ 
operating system manufactured and sold by the assignee of 
the present invention, Microsoft Corporation. Common user 
interfaces such as those found in the Microsoft "WIN- 
DOWS" operating system and other GUI-based computer 
systems typically rely on a keyboard and a pointing device 
with buttons such as a mouse or trackball for user interac- 
tion. The user points to a location on the screen where an 
action is desired, and invokes a command by pressing a 
button on the pointing device or entering a keyboard com- 
mand. 

Just as a physical desktop table can become cluttered 
when multiple documents are being viewed, a computer 
screen workspace can become cluttered when multiple docu- 
ments are displayed in multiple windows. With limited 
working space or viewing area on a computer screen, the 
computer screen workspace can quickly become cluttered if 
several applications or windows are opened at the same 
time. A number of techniques have been used for overcom- 
ing the small -screen problem presented when using com- 
puter displays. 

Four general categories of computer screen workspace 
enhancement techniques are: (1) alternating screen uses, (2) 
distorted views, (3) large virtual workspaces, and (4) mul- 
tiple virtual workspaces. Alternating -screen use is similar to 
windows in that the user can switch or change the allocation 
of screen space from one application to another. Some early 
alternating screens use techniques only allowed one appli- 
cation to be visible on a screen at a time. With distorted 
views, documents or applications appearing on the screen 
may be distorted to small icons. Here, applications are 
reduced to small pictures or icons that remind the user of the 
original window. Overlapping or stacked windows may also 
be considered a distorting technique. As noted above, when 
many windows overlap the appearance on a display screen 
may become cluttered. 

With large virtual workspaces, the set of objects, the 
documents, or applications are organized as a large virtual 
workspace which cannot all be viewed on a display screen 
simultaneously. In one example application, the data is 
arranged in two dimensions, and the user has three screens, 
one for an overview of the whole space, one for a detailed 
view of some portion of the space, and one touch screen for 
control. Head-mounted displays, which monitor user head 
and body movements to give the user a complete simulated 
three-dimensional space view, are forms of large virtual 
workspaces. A problem with a single large workspace is that 
only a limited number of things can be adjacent to any 
object. The space required for the objects and their shapes 
constrains the manner in which the space may be arranged 
and how densely packed documents or data can be pre- 
sented. Multiple virtual workspaces provide geometrically- 
oriented workspaces linked together in which a project may 
contain a number of views and, when active, covers the 
entire screen. For example, when a user maneuvers close 
enough to a port, the user is swept into another workspace 
or sub workspace. 
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The above techniques are examples of display systems system decreases the size of the window and its contents 

which try to overcome the limited workspace inherent with equivalently in the x and y dimensions, as a function of its 

computers that display multiple applications or documents assigned distance from the user in the three-dimensional 

on a two-dimensional display screen. However, these dis- space. The display system moves the window towards the 

play systems do not provide a method for displaying mul- 5 center of the space based on the relative distance between 

tiple applications presented by an operating system in three the front and the rear boundary planes. When a window on 

dimensions. Thus, there is a need in the art for a display the front plane is moved toward the rear plane, the window 

system which provides an enhanced system for displaying can snap immediately to the rear plane and appear in a 

multiple applications or windows as simulated three- reduced size or the window may move incrementally to an 

dimensional objects on > computer display screen. 10 intermediate plane or planes between the front and rear 

„ „ plane. Additionally, a window moved toward the rear plane 

SUMMARY OF THE INVENTION ^ ay continuo J; decrease in size as the ^ doy/ 

Generally described, the present invention provides a approaches the rear plane, 

three-dimensional virtual workspace for a window based User or system accessible controls are used for displaying 

display system. The display system of the present invention i5 or hanging an untransformed window on a left, right, rear, 

is an isometric display system for an operating system. The U pper, or lower boundary plane. For example, when a user 

isometric display system provides a display with monocular invokes the "perspective-transform right" command, the 

depth cues by making automatic sizing and geometric trans- display system applies a perspective transform on the rect- 

formations on two dimensional rectangles that define tradi- angular window and its contents and redisplays the rectan- 

tional windows. gular window as an appropriately sized and shaped trap- 

The isometric display system performs geometric trans- ezoid. The window as displayed appears to be rotated 90 
formation operations on rectangular windows to convey the degrees form its initial position and appears to be reposi- 
impression to the user that the windows are positioned in a tioned to fit within the right boundary plane. When a window 
three-dimensional space. The impression of depth and of is moved along or to a boundary plane, the window may be 
working in a three-dimensional space are created by trans- ^ continuously resized based on its position in the three- 
forming two-dimensional windows to appear as if the two- dimensional space or the window may snap to predefined 
dimensional windows are embedded at orientations and locations. 

positions in a three-dimensional isometric space. The iso- When the intended destination of a selected window is a 

metric display system responds to user activation of control boundary plane already containing a transformed window, 

buttons by transforming the shape and orientation of win- 3Q the selected window is placed on an adjacent plane to the 

dows consistent with the denned three-dimensional space. boundary plane that is closest to the boundary plane. Adja- 

More particularly described, the present invention pro- cent planes refer to planes that are parallel (in the three - 

vides a method of utilizing left, right, front,ba<&7Upper,?and dimensional space) to the boundary plane but that are 

lower planespThese plan^define„a_poiyhedron^d4hese positioned at selected increments away from the boundary 

planes may be referre^tozas-boimdary-planes^rThe poly- 35 plane along the normal vector from the plane towards the 

hedron surrounds^uthree^o^ the center of the virtual space. Windows selected for display on 

isometric dispky^syAtem~jo^ and a designated plane are displayed on the closest available 

position of wina5^[cohtaining:dati"from applications. The adjacent plane. This provides an ordering of displayed 

boundary planes of the virtual space are preferably orga- windows from the center of the three-dimensional space to 

nized into a rectangular prism defining left, right, front, and ^ the boundary plane for all transformed windows on the 

back walls, a ceiling, and a floor of a four-sided room as seen boundary plane. The contents of displayed windows may be 

when viewing the room from the outside through a selected generally viewed in full when the windows are hung on 

wall or user viewpoint. * different planes. Also, the windows may have focus or be 

Boundary planes may be explicitly displayed with shaded operated on when displayed or hung on a plane, 

solid surfaces orsimply as lines defining the intersection of 45 Three primary controls may be used to position a window 

planes. JHpwever, the pl anes are preferably not visible. The hung on a plane: push back, perspective-transform right, and 

front" boundary plane is typically assumed to be the front perspective-transform left. Push back causes a window to be 

surface of the display, closer to the user than the rear transformed or pushed to the foreground or background of 

boundary plane. When the boundary planes are made the display system. Perspective-transform right generally 

visible, the left and right walls, and the ceiling and floor 50 causes a window to be displayed on the right plane and 

defined by the planes narrow in dimension as viewed from perspective-transform left generally causes a window to be 

the front to back of the space defined on the display screen. displayed on the left plane. Alternatively, a single traosfor- 
Windows may be oriented and displayed in the boundary^ mation control for transforming windows to the various 

planes. planes may be implemented. Upon selection of the single 

Windows oriented in the front boundary plane are dis- 55 transformation control, the display system provides a pointer 

played without transformation, and appear as standard rect- to a selected plane. The pointer may be operative to point to 

angular window, as in traditional windowing systems. When a plurality of predefined planes. The pointer may be a 

a transformation command is entered by a user or from a direction arrow. Generally, the pointer is provided when the 

program, a window is transformed and moved to the selected cursor control is activated or pulled in the direction of a 

plane of the display system to create the perception of depth, so predefined plane while the cursor is positioned on the 

A window selected for display on the rear plane appears as transformation control. The user may release the cursor 

a rectangular window that has been pushed back in the control to initiate the transfer or transformation of the 

three-dimensional space. A window designated for another window. 

plane is rotated 90 degrees and appears pushed flat against The operating system utilized in connection with the 

the designated plane. 65 present invention receives data from an operating system, an 

For generating the illusion of moving the windows away application or other source for display in a window. The data 

from the user in the three-dimensional space, the display may be stored by the operating system in the same manner 
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as in rectangular windows. Before displaying the data in a FIG. $a, Sb, and 5c illustrate another manipulation of 

window on the display screen, the operating system scales displayed windows in accordance with the present inven- 

the shape of the window to fit the selected plane and the data tion. 

presented in the window. The scaling of the window's shape piG. 6a, 6b, and 6c illustrate another manipulation of 

and the scaling of the data presented in pixel coordinates 5 displayed windows in accordance with the present inven- 

within the window creates a three-dimensional appearance tion. 

on the display screen. FIG ? ^ 7 ^ and Jd mustrate aQ alternate embodiment 
Transformation matrices may be used in the operating 0 f a control mechanism for manipulation of displayed win- 
system to transform or scale a window to the selected plane. dows in accordance with the present invention. 
When a user indicates that a window should be displayed on io p[G ^ &nd $d m ^ ai& m embodiment 
a particular plane, both the coordinates of the window and Qf a mechanism for mampulation of displayed win- 
data contained in the window are multiplied by a transfer- dows ^ accordance with ihQ nt invenlion . 
ma tion matrix to fit the window in the area defined by the m „ , Af .„ . t 
selected plane and transform the data content of the window ' 9u a u nd 9b ^ strate « aUernate emboclunent of a 
By transforming the window and data, depth perception for 15 control mechanism for manipulation of displayed windows 
the window is created. m accordance Wllh me P resent ^^n. 

Hie present invention may also provide a method of F[G - 10fl L and 10 f> i^trate alternate embodiment of a 

dynamically sizing the cursor as a function of the location of 00111101 mechanism for manipulation of displayed windows 

the cursor on the display screen. The present invention may in accordance with the present invention, 

determine the position of the cursor on the display screen, 20 F[G * 11 illustrates lines demarcating areas for display of 

and resize the cursor based upon the location of the cursor windows in accordance with the preferred embodiment of 

on the display screen. The size of the cursor may change tne P resen l invention. 

when the cursor crosses over a border of a window. The size FIG. 12 illustrates a non-planar bounding surface for 

of the cursor may specifically change relative to the cursor defining a three-dimensional workspace. 

location on a window to reflect the depth of that location in 25 FIG. 13 illustrates multiple isometric spaces constructed 

three-dimensional space defined by the transformed win- according to an embodiment of the invention. 

dows. This serves to give feedback to the user about the 14 illustrates an alternate view of the multiple 

location of the cursor io three-dimensional space by gener- isometric spaces. 

ating the perception that the cursor resides on or jumps to FIG. 15 illustrates the method of dynamically sizing the 

planes defined by the transformed windows. curS0f of ^ present 

Thus it is an object of the present invention to provide an F[GS Ua and Ub are flow ^ ff9m& showing ^ steps 

enhanced display system for displaying applications and ^1^^ in the preferred embodiment of the present 

documents in a computer system. invention 

It is another object of the present invention to manipulate 35 
the displayed windows to create a virtual three-dimensional DETAILED DESCRIPTION 
workspace on the monitor of the computer system. Referring now to the drawings, in which like numerals 
It is another object of the present invention to provide a illustrate like elements throughout the several views, FIG. 1 
display system for simultaneously displaying multiple appli- illustrates the basic components of a computer system 10 
cations and objects managed by the operating system as a 40 use d in connection with the preferred embodiment of the 
consistent three-dimensional operating environment. present invention. FIG. 1 illustrates a graphical user inter- 
It is another object of the present invention to transform face 12 on a computer display screen 15 of a display monitor 
the shapes and contents of a displayed window to create a 14 of the computer system 10. The graphical user interface 
perspective view of the window and its contents. of the display system is preferably implemented as part of 
It is another object of the present invention to partition the 45 the operating system to provide a three-dimensional envi- 
display screen of a display monitor into mu^ple^lanes for ronment for displaying and managing the window objects of 
displaying selected windows in* a preselected forhi. the operating system. The present invention enables cutting 
It is another object of the present invention to dynamically and pasting, moving objects, and other functions as gener- 
size a cursor relative to the location of the cursor on the aU y performed by an operating system while providing 
display screen 50 visual depth cues displayed by the operating system consis- 

These and otheapbjerts S featufe^ and advantages, of the ten ' ori f tatio ° f d P osition of t hc *j ec ' ^ 

present <9R6Bffiff$ff becorrZapparent fiom'Teading the b ° U " darie ? 1 definm f * pH> " dBlg ?" 

following description in conjunction with the accompanying d fP th , cues - Additionally, sounds such as beeps and warning 

d - signals, generally associated with the interaction of a user 

55 with a computer system, may appear, using stereo sound, to 

BRIEF DESCRIPTION OF THE DRAWINGS emanate from the window upon which a user is operating in 

^ . , , - the three-dimensional space. By varying the level of the 

FIG. 1 is a diagram showing the basic components of a and/Qr from wMch ^ emanat the 

computer system used m connection with the preferred SQUnd f ^ emanate frQm a 

embodiment of the present invention. m MxhQU%h ^ pngcjA mvention fs described m conjunctioo 

FIG. 2 is a block diagram showing the basic components ^th a conventional desktop computer, it will be appreciated 

of the computer system used in connection with the pre- mat me present invention may be utilized in other types of 

ferred embodiment of the present invention. computer systems that use a window based display system. 

FIG. 3 illustrates a detailed view of the display screen The graphical user interface system 12 is implemented with 

generated in accordance with the present invention. 65 a computer 16 connected to the display monitor 14. The 

FIG. 4a, 4b, and 4c illustrate a manipulation of displayed computer 16 receives input data from a conventional key- 

windows in accordance with the present invention. board 20 via an input tine 18. Cursor keys on the keyboard 
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20, a mouse 21, trackball, or other pointing device may be 
used to move a cursor 23 on the display screen 15 for 
selection of various options. It should be appreciated by 
those skilled in the art that text entry or command input also 
may be accomplished by using a touch-sensitive pad or 5 
handwriting recognition device in conjunction with software 
for recognizing the corresponding input signals as selection 
signals. 

Referring to FIG. 2, the hardware components of the 
computer 16 are illustrated. For simplicity of the drawings, 10 
many components of a standard computer system have not 
been illustrated such as address buffers, memory buffers and 
other standard control circuits because these elements are 
well known and illustrated in the prior art and are not 
necessary for the understanding of the present invention. A 15 
computer program used to implement the various steps of 
the present invention is generally located in the memory unit 
20, and the processes of the present invention are carried out 
through the use of a central processing unit (CPU) 22. Those 
skilled in the art will appreciate that the memory unit 20 is 20 
representative of both read-only memory and random access 
memory. The CPU 22 is typically implemented as a single - 
chip microprocessor, such as the models 80386 or 80486 that 
are available from Intel Corporation, Santa Clara, Calif. The 
CPU 22, in combination with computer software, such as an ^ 
operating system 25 and application programs 27, controls 
the operations of the computer system 10. The operating 
system software 25, in conjunction with application pro- 
grams 27, controls the allocation and usage of hardware and 
software resources such as memory, CPU time, disk space, 30 
and peripheral devices. It should be appreciated that many 
window based operating systems exist such as Microsoft 
"WINDOWS", a UNIX (e.g. X-windows) operating system, 
or a Macintosh operating system. The display routines are 
encoded in a graphical user interface display layer in the 35 
operating system. The system bus 24 supports communica- 
tions of control, address, and data signals between the CPU 
22 and the remaining components of the computer system 
10. The memory unit 20 and the CPU 22 are connected by 
a computer system bus 24 designed to provide an electrical ^ 
interface between computer system components. The pro- 
cesses implemented by the CPU 22 may be communicated 
as electrical signals along the computer system bus 24 to an 
input/output device 26, such as display monitor 14. The 
display monitor 14 provides a visual display of computer 45 
generated graphics originating from the processes imple- 
mented by the CPU 22. 

In this detailed description, numerous details are provided 
such as computer display system elements, object 
definitions, display formats, sample data, etc., in order to 50 
provide an understanding of the invention. However, those 
skilled in the art will understand that the present invention 
may be practiced without the specific details. Well-known 
circuits, programming methodologies, and structures are 
utilized in the present invention but are not described in 55 
detail in order not to obscure the present invention. 

Certain of the descriptions which follow are presented in 
terms of display images, processes, and symbolic represen- 
tations of operations of data bits within the computer's 
memory. As will be known to those skilled in the program- eo 
ming arts, these process descriptions, class descriptions, 
messages, notification descriptions, and graphic displays are 
the means used by those skilled in the art of computer 
programming and computer construction to convey teach- 
ings and discoveries to others skilled in the art, 65 

For purposes of this discussion, a process or method is 
generally a sequence of computer- executed steps leading to 
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a desired result. These steps require physical manipulations 
of physical quantities. Usually, though not necessarily, these 
quantities take the form of electrical, magnetic, or optical 
signals that are capable of being stored, transferred, 
combined, compared, or otherwise manipulated. It is con- 
ventional for those skilled in the art to refer to these signals 
as bits, data, values, elements, symbols, characters, images, 
terms, numbers, or the like. It should be kept in mind, 
however, that these and similar terms should be associated 
with appropriate physical quantities inside the computer and 
that these are merely convenient labels applied to these 
physical quantities that exist within the computer. 

It should also be understood that manipulations within the 
computer are often referred to in terms such as adding, 
comparing, moving, etc., which are often associated with 
mental operations performed by a human operator. It must 
be understood that involvement of a human operator is not 
necessary in the present invention because the operations 
described herein are machine operations performed in con- 
junction with a human operator or user that interacts with the 
computer. The machines used for performing the operation 
of the present invention include general purpose digital 
computers or other similar computing devices. 

The present invention relates to methods, steps, or opera- 
tions for a computer which processes electrical or other 
physical signals to generate desired physical signals and 
display results and interactions. As illustrated in FIG. 1, the 
present invention also relates to apparatus 10 for performing 
these operations. This apparatus 10 may be especially con- 
structed for the described purposes. 

Furthermore, it should be understood that the programs, 
processes, objects, etc., described herein are not related or 
limited to any particular computer or apparatus. Rather, 
various types of general purpose machines may be used with 
programs constructed in accordance with the teachings 
herein. Similarly, it may prove advantageous to construct 
specialized apparatus to perform the method steps described 
herein by way of dedicated computer systems with hard- 
wired logic or programs stored in nonvolatile memory such 
as read only memory. 

Referring again to FIG. 1, the programming techniques 
and manner of handling windows in a computer system will 
be discussed, Those skilled in the art will understand and 
appreciate that operating systems for displaying and utiliz- 
ing windows may be implemented by using object-oriented 
programming languages and techniques. Object-oriented 
programming techniques are utilized by those skilled in the 
art of computer programming because they provide a pow- 
erful way to view the interaction of a user with a computer 
system. For example, one particular "object" that people 
often work with on computer screens is called a window. 
Referring to FIG. 1, a window 30, as discussed above, is a 
rectangular area on the display screen 15 that is used to 
display information or receive commands or data from the 
user. The display screen 15 itself may be considered a 
window into the computer 16. The window 30 receives user 
input from a keyboard 20, mouse 21, or pen, and displays 
graphical output upon which a user may interact. The 
display screen 15 thereby serves as the workspace for the 
computer. A computer program is responsible for generating 
the window 30 and for responding to user interactions with 
the window 30. The window 30 often contains the computer 
program's title bar 32 across the top of the window 30, 
menus, sizing borders, and perhaps other objects. 

The window 30 may contain additional, smaller windows 
called "child windows", which are subspecies or subclasses 
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of windows. Child windows may take the form of push 
buttons, radio buttons, check boxes, text entry fields, list 
boxes, and scroll bars, and the like. These objects when 
displayed on the computer screen may be pressed or clicked 
or scrolled. A user sees a window as an object on the screen, 
and interacts directly with the object by pushing buttons 34, 
or scrolling a scroll bar, or tapping a handle displayed in the 
window 30. Buttons 34 may control a function for the 
particular window. 

A window generated by a computer program receives user 
input in the form of messages to the window. The messages 
are generated by an event handling routine that is responsive 
to user input such as tapping, clicking, or dragging a cursor 
with a stylus or mouse. 

The following is a brief example of an object-oriented 
programming methodology for resizing a window utilized in 
the Microsoft "WINDOWS" graphical user interface oper- 
ating system. Consider a word processor application com- 
puter program that displays text in a window. The window 
may include a "size box", which is a control located at the 
lower right-hand corner of the window. If selected and 
dragged, the size box causes the size of the window on the 
display screen to become larger or smaller. If the program's 
window is resized, the word processing program application 
program will reformat and move the text on the display 
screen to fit within the window when the window is resized. 

The computer's operating system program generally 
handles the details of resizing the window, and the applica- 
tion program running under control of the operating system 
program responds to the system function of resizing the 
window. The word processing program "knows" when its 
window is resized because a message is passed to it indi- 
cating a resizing operation, and the application program 
responds by repositioning the text within the available 
window. In the "WINDOWS" operating system, when a user 
resizes a window, the operating system sends a message to 
the application program indicating the new window size. 
The program then adjusts the contents of its window to 
reflect the new size. 

The passing of a message to a program is tantamount to 
making a call to a function within the program. Such a 
function call can be made to the operating system or to an 
application program. Those skilled in the art will understand 
that such mechanisms are how a program opens a disk file, 
for example, or scrolls text from an area that is "off screen" 
into view within the window. An operating system may 
make calls to a program by passing it messages, in a similar 
manner that the application program makes a call to the 
operating system. 

Every window object that a program creates has an 
associated window procedure (also called a "method"). This 
window procedure or method is a function that could be 
either in the program itself or in a linked library of appro- 
priate routines that can be executed upon a call. The "WIN- 
DOWS" operating system sends a message to a window by 
calling a window procedure. The window procedure does 
processing based on the message passed to it and then 
returns control .to the operating system, possibly with a 
"notification" or other response. (A "notification" is merely 
a message passed back to a calling program, and usually 
indicates status information or the results of an operation.) 

A window displayed on a display screen is based on what 
is known as a "window class" of objects. The window class 
of objects identifies a particular procedure that processes 
messages passed to that window. The use of a window class 
allows multiple windows to be based on the same window 
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class, and thus use the same window procedure. For 
example, control buttons in programs that utilize the "WIN- 
DOWS" operating system are based on the same window 
class. A window class is associated with a window procedure 

5 that processes messages to all button windows. 

In object-oriented programming, an "object" is a combi- 
nation of computer program code and data. A window is a 
type of object. The computer program code carries out tasks 
associated with the object. The data is the information 

iq retained by the window procedure and information retained 
by the "WINDOWS" operating system, for each window or 
window class that exists. 

A window procedure processes messages to the window. 
Often these messages inform a window of a user input from 

15 a keyboard, mouse, or pen. This is how a child window that 
is a button "knows" that it is being "pressed," or an icon 
"knows" that it is being dragged. Other messages tell a 
window when it is being resized or when the surface of a 
window needs to be repainted on the monitor. 

20 When a program begins execution in such an operating 
system as the "WINDOWS" operating system, the operating 
system creates a "message queue" for the program. This 
message queue stores messages to all the various windows 
a program may create. The program includes a portion of 

25 program code called the "message loop" to retrieve the 
messages from the queue and dispatch them to the appro- 
priate window procedure. Other messages may be sent 
directly to a window procedure without being placed in the 
message queue. 

30 Three-Dimensional Window Operation and Control 

Referring to FIG. 3, a more detailed view of the graphical 
user interface 12 of the display screen 15 of the present 
invention is shown. The graphical user interface 12 provides 
depth cues in a manner to yield a three-dimensional view of 

35 the windows and/or objects displayed on the display screen 
15. A window is considered to be transformed when three- 
dimensional depth cues are provided to the window. Thus, a 
simulated three-dimensional display of the windows is pro- 
vided. By providing three-dimensional graphics to the 

40 windows/objects created and managed by an operating 
system, the computer workspace is effectively enlarged for 
a standard graphical user interface consisting of windows, 
icons, menus, and pointer (WIMP). The display screen may 
be considered to have an x, y, z coordinate system with an 

45 x-axis 314, a y-axis 310, and a z-axis 312. The three- 
dimensional virtual workspace displays the windows and 
objects of the operating system so that the windows and 
objects appear to project along the z-axis 312 into the 
display screen 15. Each displayed window has its own set of 

50 buttons and controls as in conventional window environ- 
ments including transformed windows. 

The display or positioning of the windows on the display 
screen 15 is within predefined boundaries that may. be 
considered a space or open room. The open room can be 

55 viewed from the outside, with the room having front, back, 
top, bottom, left, and right walls. As used herein, "walls" of 
a three -dimension room space may be referred to as planes. 
The front plane 38 shown containing the window 40 pro- 
vides a view which is similar to a window as typically 

60 presented in conventional windowing environments in two 
dimensions. In conventional windowing environments, the 
windows and data are shown on a single front or forward 
two dimensional plane. However, unlike conventional win- 
dow systems, a back plane 44 and other planes may be 

65 utilized to display a window 46 as illustrated. 

The back plane 44 displays a window that is generally the 
same shape (rectangular) as a window on the front plane 38 
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except that the window 46 displayed on the back plane 44 is 
zoomed back or set at a distance (along the z-axis 312) from 
the front plane 38. Thus, the windows appearing or "hang- 
ing" on the backplane 44 have smaller dimensions and thus 
appear farther away than the windows appearing on the front 
plane 38. In addition to the window 46 appearing smaller 
than the window 40 on the front plane 38, the text within the 
window 46 on the back plane is also reduced in scale to 
further give the appearance of distance along the z-axis 314. 

As noted above, the left and right planes 50 and 50 are 
analogous to the sides or the walls of a room or box. As 
illustrated, a window 58 may be positioned on or "hung" on 
the right plane 50. Similarly, a window 52 may be hung on 
the left plane 50. Windows hung on either the left plane 50 
or the right plane 56 are considered to be in "perspective." 
When a window is in perspective, the edge 53 of the window 
that is farthest away from the user's view, as looking along 
the z-axis 314 into the display screen 15, is smaller than the 
edge 55 that appears closer to the user. Although not 
illustrated with windows displayed thereon in FIG. 3, bot- 
tom or floor planes 106, and top or ceiling planes 102 may 
be used for hanging windows at the bottom or top of the 
screen, respectively. Windows appearing at the top or bot- 
tom of the display screen 15 would likewise be displayed in 
perspective at the bottom or top of the screen, similar to the 
windows 52 and 58 displayed on the left plane 50 and the 
right plane 56, respectively. 

When an application program is initially opened, a win- 
dow is created and attached to the front plane 38. The 
window will remain on the front plane 38 until the user 
positions the window on another plane by use of a transfer 
or transformation control, such as the perspective-transform 
left button 66, perspective-transform right button 68, or 
pushback button 64. The current plane will become part of 
the profile information for the application so subsequent 
invocations of the program will result in a window being 
created and attached to the plane that the window was 
attached to when the application was last shut down. For 
example, if the main window of an application is attached to 
the left plane 56, when the user closes the application using 
the exit command on the file menu of the application, the 
next time the user runs the application, the application's 
main window will appear on the left plane 56 because that 
is where the main window was when the application was 
terminated. 

When a user resizes a window or perspective-transforms 
a window in accordance with the present invention, the 
computer application handles the resizing operation as the 
program would in a conventional window environment. 
However, the operating system provides the appropriate 
scaling for the shape, size, and contents of the window to fit 
on the appropriate plane. Thus, no specialized programming 
is required for computer applications designed to operate in 
conventional window-based operating systems. 

In conventional window display systems, the operating 
system specifies a destination rectangle which specifies the 
location in the frame buffer for displaying the window 
containing the computer application data. The destination 
rectangle is generally specified in pixel coordinates. The 
operating system utilized in connection with the present 
invention receives data that is created to be stored in a 
destination rectangle in a frame buffer then scales the data to 
dimensions that provide a destination area yielding a three- 
dimensional depth or perspective appearance on the display 
screen. This destination area is given in pixel coordinates. 

When the operating system and CPU detect that one of the 
control buttons has been selected, the operating system 
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notifies the appropriate window object, and it then uses a 
series of notifications (a) to release itself from the plane that 
it is currently attached to and (b) to attach itself to a different 
plane based on the control that was selected. Transformation 

5 matrices, as known to those skilled in the art, are applied 
during rendering of the image. The transformation matrices 
are applied to the drawing commands that an object sends to 
the graphics engine of the computer system. The matrices 
are standard 4x4 transformation matrices known to those 

10 skilled in the art. 

With continuing reference to FIG. 3, the control buttons 
34 for controlling the positions of the display windows on 
the various p^anesof the display system areshown. A system 
menu button 60,-a"tiUe r b'afr62, a push back button 64, a 

15 perspective-transform left button 66, and a perspective - 
transform right button 68 are shown along the top of window 
40. The same control buttons are displayed in the same 
position on the window 46 appearing on the back plane 44 
and in a left-to-right arrangement as illustrated on the 

20 window 40. When a window is attached or hung on the left 
plane 50 or right plane 56, the controls 34 are arranged top 
to bottom as illustrated on windows 58 and 52. Also, the title 
bar 62 is realigned so that when the title bar 62 is attached 
to either the right plane 56 or the left plane 50, the text runs 

25 from top to bottom. The function of the controls of the 
display system are as follows. 

The system menu accessed by the system menu button 
may provide a menu access to the functionality provided by 
the pushback button 64, perspective-transform left button 

30 66, and perspective -transform right button 68 as well as 
access to functions that move and resize a window. The 
functionality of moving and resizing is the same as the 
system menu in the current Microsoft Windows GUI, The 
title bar 62 contains the textual representation of the name of 

35 the "owning" window. The term "owning" generally means 
the object containing a specified item or the object to which 
the item pertains. The name appearing in the title bar 62 can 
be changed by the operating system program by means of a 
message to the owning window when a user makes the 

40 appropriate change. The text is re -oriented, such as left- to- 
right or top-to-bottom, based upon the plane on which the 
window is displayed. 

By selecting the push back button 64 of a particular 
window, the window is displayed or moved to either the 

45 front plane 38 or back plane 44 depending upon which of the 
planes the window is currently located. If the push back 
button 64 is selected and the window was on front plane 38, 
then the window is zoomed backward and attached to the 
backplane 44. If the window is currently located on the back 

50 plane 44 when the push back button 64 is selected, then the 
window is zoomed forward and attached to the front plane 
38. Similarly, if the push back button 64 is selected on a 
window 58 that is located on a side plane, the window is 
pushed back to either the front plane 38 or the back plane 44 

55 depending on whether the window was previously attached 
to the front plane 38 or the back plane 44. When the push 
back button 64 is selected and if the window 58 was 
previously attached to the front plane 38, then the window 
58 is pushed back to the back plane 44 and vice versa. When 

60 the push back button is selected and if the window 52 was 
previously attached to the back plane 44, then the window 
52 is pushed back to the front plane 38 and vice versa. 

When the perspective -transform left button 66 is selected 
for a particular window, the window is placed on the left 

65 plane. If the window is currently attached to either the front 
plane 38 or the back plane 44, then the operating system 
causes the window to be removed from the current plane and 
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displayed on the left plane. When a window is selected to be 
transferred to the left plane 50, the current plane containing 
the window is saved as the last saved plane for that selected 
window to be referenced when the perspective-transform 
controls are subsequently selected for the window. If the 5 
window is currently attached to the left plane 50 when the 
perspective-transform left button is selected, the operating 
system 25 removes the current window from the left plane 
and attaches the current window to the last saved plane. For 
example, when a window is attached to the front plane and 10 
the perspective-transform left button is selected, the window 
is attached to the left plane, and the front plane position is 
saved as the last saved plane. Selecting either the 
perspective-transform left or perspective-transform right 
button on this window will result in the window being is 
detached from the left plane and returned to the front plane. 
If the window had been attached to the back plane prior to 
the perspective-transform left button being selected, the 
window would be returned to the back plane. The system 
does not save the left plane position when a window is 
removed from the left plane because the perspective- 
transform left button positions the window on the left plane. 

When the perspective-transform right button 68 is 
selected for a particular window, the window will be placed 
on the right plane. If the window is currently attached to 
either the front plane 38 or the back plane 44, then the 
operating system causes the window to be removed from the 
current plane and displayed on the right plane. When a 
window is selected to be transferred to the right plane 56, the 
current plane containing the window is saved as the last 
saved plane for that selected window to be referenced when 
the perspective-transform controls are subsequently selected 
for the owning window. If the window is currently attached 
to the right plane 56 when the perspective-transform right 
button 68 is selected, the operating system 25 removes the 
current window from the right plane and attaches the current 
window to the last saved plane. For example, when a 
window is attached to the front plane 38 and the perspective - 
transform right button 68 is selected, the window is attached 
to the right plane, and the front plane position is saved as the 
last saved plane. Selecting either the perspective-transform 
left or perspective-transform right button on this window 
will result in the window being detached from the right plane 
56 and returned to the front plane 38. If the window had been 
attached to the back plane prior to the perspective-transform 
right button 68 being selected, the window would be 
returned to the back plane. The system does not save the 
right plane position when a window is removed from the 
right plane because the perspective-transform right button 
positions the window on the right plane. so 

Basic controls 60, 64, 66, and 68 utilized with the present 
invention are invoked with a mouse selection device in the 
same manner as the controls are selected in a conventional 
window environment. Also, the windows operating within a 
system of the present invention may be "dragged" to new 55 
positions as in conventional window applications. By click- 
ing and holding the primary mouse button while the cursor 
23 is in the title bar of a window, the window can be dragged 
to a new location. The window may be dragged to a new 
location when it is on any of the planes. The window 60 
movement and size may be limited to the extents of the 
defined planes. 

Referring again to FIG. 1, windows may be stacked 
behind one another as illustrated with windows 70 and 72. 
Window 72 appears behind window 70. Because these 65 
windows are stacked, an order for these windows is estab- 
lished. The ordering of the windows is termed the "z" 
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position. A list, termed the z-order list, indicates the order or 
spacing of windows as the windows appear on a plane of the 
display screen. A basic stacking procedure places a window 
into the first available slot in the z-order list. When placing 
a window in the first available slot, a newly perspective- 
transformed window may disappear behind windows that 
are already attached to the plane rather than appearing in 
front of the windows that are currently displayed on the 
plane. For example, when the intended destination of a 
selected window is to a boundary plane already containing 
a transformed window, the selected window is placed on the 
adjacent plane to the boundary plane that is closest to the 
boundary plane, such as plane 71 containing window 72. 
Adjacent planes refer to planes that are parallel (in the 
three-dimensional space) to the boundary plane but that are 
positioned at selected increments away from the boundary 
plane along the normal vector from the plane towards the 
center of the virtual space. Windows selected for display on 
a designated plane are displayed respectively to the closest 
available adjacent plane. This provides an ordering from the 
center of the three-dimensional space to the boundary plane 
for all transformed windows on the boundary plane. When 
placing multiple windows on the left and right planes, the 
windows are preferably cascaded slightly down the wall and 
slightly toward the user. The contents of displayed windows 
may be generally viewed in full when the windows are hung 
on different planes. Also, the windows may have focus or be 
operated on when displayed or hung on a plane. 

Each plane in the system allocates an array to contain 
pointers to windows. This array is initialized so that all of the 
storage locations are set to zero. When a new window is 
attached to a plane the array of window pointers is searched 
and advancing until the first slot in the array that is zero is 
found. Once an empty position is found in the array, the new 
window pointer is stored at the location. When a window is 
detached from a plane, the location in the array that con- 
tained the pointer to the window is set to zero. When a plane 
is notified that it needs to paint, it iterates through its window 
pointer array starting at zero. The plane has a member 
variable that is the number of windows that are currently 
attached to the plane. The painting loop terminates when it 
either: (a) reaches the end of the array; or (b) determines that 
it has painted all of the windows that are currently attached 
to the plane by incrementing a local variable every time that 
it finds a window in the array and comparing this local 
variable to the member variable that is the count of the 
windows currently attached to the plane. It is to be appre- 
ciated by those skilled in the art that the specific z-position 
of a perspective-transformed window may include various 
ordering schemes known by those skilled in the art. 

Referring to FIG. 4A, 4B, and 4C, the operation of the 
push back button 64 as applied to a window is illustrated. 
Referring to FIG. 4A, a window 410 is shown displayed on 
the front plane 38. When the push back button 64 is selected 
within the window 410 by the cursor 23 as shown, the 
window 410 and its data content 415 is transformed or 
moved to the back plane 44 as illustrated in FIG. 4B. After 
the push back selection was initiated in FIG, 4A, the final 
position of the window 410 is shown in FIG. 4C. The 
window 410 shown in FIG. 4C has been fully zoomed 
backward (set at a distance along the z-axis) to the back- 
ground where the window 410 and its contents 415 have 
been rescaled to appear at distance from the front plane 38. 

Referring to FIGS. 5A, SB, and 5C, the operation of the 
perspective-transform right button 68 is illustrated. FIG. 5A 
shows a window 510 displayed on the backplane 44. When 
the perspective- transform right button 68 is selected within 
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the window 510 by the cursor 23 as directed by the user, the of the window transform button 710 when a pushback or 
window 510 is transformed or moved to the right plane 56 zoom out operation is selected by a user. Referring to FIG. 
as illustrated in FIG. 5B. After the perspective-transform 7A, the cursor 23 is shown positioned on the window 
right selection was initiated in FIG. 5A, the final position of transform button 710. The user has selected the zoom out 
the window 510 is shown in FIG. 5C The window 510 and 5 function of the window transform button 710. The user 
its data content 515 shown in FIG. 5C has been fully makes the selection by positioning the cursor 23 on the 
transformed to the right plane 56 and given a shape that window transform button 710 and by clicking on the win- 
creates the appearance of a perspective rendering, with the dow transform button 710 and dragging the cursor 23 
left side of the window 510 narrowing as it approaches a upwards in the direction of the back plane 44 as indicated by 
selected point or the center point of the display screen 15 as 10 the plane direction arrow 730. The direction indicated by the 
the distance along the z-axis increases away from the user. plane direction arrow 730 points toward the zoom out or 

Referring to FIGS. 6A, 6B, and 6C, the operation of a back plane 44 position. Thus, the user is provided with an 

perspective-transform below button 601 is illustrated. FIG. indication of the computer's interpretation of the user's 

6A shows a window 610 displayed on the plane 38. When selection. If the user desires to have the window 722 zoomed 

the perspective-transform below button 601 is selected 15 back to the back plane 44, a user simply releases the 

within the window 610 by the cursor 23 as directed by the selection mechanism of the cursor control while the direc- 

user, the window 610 is transformed or moved to the bottom tion arrow 730 is pointing in the direction of the zoom back 

plane 106 as illustrated in FIG. 6B. After the perspective- position. Upon release of the selection mechanism of the 

transform below selection was initiated in FIG. 6A, the final cursor control, the window 722 is transformed or zoomed 

position of the window 610 is shown in FIG. 6C. The 20 back to the back plane position as illustrated in FIG. 7B. 
window 610 and its data content 615 shown in FIG, 6C has Similar to the method by which the window 722 is 

been fully transformed to the bottom plane 106 and given a positioned on the back plane 44, the user may zoom forward 

shape that creates the appearance of a perspective rendering or transform the zoomed back window 722 of FIG. 7C by 

with the far side of the window 610 narrowing as it ■ positioning of the cursor 23 on the title bar 725 and then 

approaches a selected or center point of the display screen 15 25 clicking and dragging the cursor 23 upwards to produce the 

as the distance along the z-axis increases. two -angled direction lines 734. The direction lines 734 

The push back button 64, the perspective-transform left indicates that the computer interprets the user's selection as 

button 66, the perspective-transform right button 68, and the a desired transformation to the front plane 38. If the user 

perspective-transform below button 601 may be used to accepts this interpretation by the computer, the user releases 

transform a nontransformed window into a perspective 30 the selection mechanism of the cursor control which then 

transformed window or converting a perspective trans- causes the window 722 to zoom forward or fling to the front 

formed window into a nontransformed window by selecting plane position as illustrated in FIG. 7D. 
any one of the individual controls. An alternative method Referring to FIGS. 8A and 8B, the control method for 

that utilizes a one button control process for transforming positioning a window 822 on the right plane 56 using the 

the windows to the multiple views within the operating 35 window transform button 710 is shown. In FIG. 8 A, a cursor 

system. The one button control for determining the position 23 is positioned on the window transform button 710, and 

of windows on the various walls or planes is discussed in the user clicks and drags the cursor toward the right plane 

connection with FIGS. 7A-10C. The one button control 38. When a user clicks and drags the cursor 23 towards the 

discussed in FIGS. 7A-10C utilizes a click-and-drag type right plane 38 while the cursor 23 is positioned on the 

functionality to designate a window to be transformed. 40 window transform button 710, a plane direction arrow 830 

Referring to FIGS. 7A-10B, the operation of a single points towards the right plane 56. As discussed above, if the 

multiple control window transform button 710 is illustrated. user desires to indicate or accept the computer's interpreta- 

A user may guide or place a cursor 23 on the window tion of the user's selection as indicated by the direction 

transform button 710 on the tool bar of the nontransformed arrow 830, the user simply releases the selection mechanism 

window 722. A user may then press the selection mechanism 45 of the cursor control, and the window 822 is positioned on 

of the cursor control, such as a control button on a mouse, the right plane 56 as illustrated in FIG. 8B. Referring to FIG. 

and hold the mouse control button down and drag it in the 8C, the method by which a user repositions the window 822 

direction of the plane on which the user wants the window on the front plane 38 is illustrated. In order to reposition the 

displayed. As the window transform button 710 is dragged, window 822 on the front plane 38, the user clicks on the 

an arrow is displayed in the direction of a drag motion 50 window transform button 710 and releases the selection 

indicated by the user. By displaying the direction arrow, the mechanism of the control mouse. Upon release of the 

user is provided with feedback or given an indication of the selection mechanism of the cursor control, the window 822 

computer's interpretation of the user's selection. If the user is repositioned on the front plane 38 as illustrated in FIG, 

agrees with or accepts the plane selection indicated by the 8D. A direction arrow is not displayed when the window is 

direction arrow generated that points in a direction of the 55 transformed from a perspective view. As discussed above in 

plane on which the window will be displayed, the user may connection with FIGS. 4A-6C, the last plane information is 

release the mouse control button when the plane selection saved. 

arrow is displayed. The window will then be displayed on Referring to FIGS. 9A and 9B, the control method for 

the plane indicated by the selection arrow. The window positioning a window 922 on the left plane 50 using the 

appears to "fling" to the plane or wall pointed to by the plane 60 window transform button 710 is shown. In FIG. 9 A, a cursor 

direction arrow. By clicking on the window transform button 23 is positioned on the window transform button 710, and 

710 in a window that has already been perspective the user clicks and drags the cursor toward the left plane 50. 

transformed, the window is transformed back to a standard When a user clicks and drags the cursor 23 towards the left 

window and is displayed on the front plane. plane 50, while the cursor 23 is positioned on the window 

Referring to FIGS. 7A, 7B, 7C, and 7D, the operation of 65 transform button 710, a plane direction arrow 930 points 

the single window transform button 710 is illustrated. The towards the left plane 50. As discussed above, if the user 

example of FIGS. 7A, 7B, 7G, and 7D shows the operation desires to indicate or accept the computer's interpretation of 
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the user's selection as indicated by the direction arrow 930, 
the user simply releases the selection mechanism of the 
cursor control, and the window 922 is positioned on the left 
plane as illustrated in FIG. 9B. The method by which a user 
repositions the window 922 on the front plane 38 is accom- 
plished in the same manner discussed in connection with 
FIGS. 8C and 8D. 

Referring to FIGS. 10A and 10B, the control method for 
positioning a window 1022 on the bottom plane using the 
window transform button 710 is shown. In FIG. 10A, a 
cursor 23 is positioned on the window transform button 710, 
and the user clicks and drags the cursor toward the bottom 
plane 106. When a user clicks and drags the cursor 23 
towards the bottom plane 106, while the cursor 23 is 
positioned on the window transform button 710, a plane 
direction arrow 1030 points towards the bottom plane 1030. 
As discussed above, if the user desires to indicate or accept 
the computer's interpretation of the user's selection as 
indicated by the direction arrow 1030, the user simply 
releases the selection mechanism of the cursor control, and 
the window 1022 is positioned on the bottom plane 106 as 
illustrated in FIG. 10B. The method by which a user 
repositions the window 1022 on the front plane 38 is 
accomplished in the same manner discussed in connection 
with FIGS. 10C and 10D. 

Planes and Boundaries for Three-Dimensional Windows 

As noted above, window information is scaled and con- 
fined to certain areas on the display screen. Referring to FIG. 

11, a system for defining the perspective transform based 
environment and for providing and defining the positions in 
which a window may be displayed is illustrated. A virtual 
isometric space system 80 provides a right plane quadrilat- 
eral 84, left plane quadrilateral 88, and back plane and front 
plane rectangles 92 and 96, respectively. The quadrilaterals 
and rectangles form the borders of a rectangular-prism or 
polyhedral isometric workspace. It should be appreciated 
that more or less complex surfaces may be used. The 
boundaries defining the rectangles or quadrilaterals are 
consistent with the predefined isometric space. The walls, 
ceiling and floor may be explicitly displayed on the display 
screen at the user's option. The plane based system con- 
strains window shapes and sizes to the shape and sizes of the 
defined polygons. In this example, two walls, the right plane 
quadrilateral 84 and the left plane quadrilateral 88, and two 
depth planes, the front and back plane rectangles 92 and 96, 
respectively, define the three-dimensional space operative to 
display windows. It should be noted that the top and bottom 
planes 102 and 106 could also be used to display and 
transform windows. It should be appreciated three- 
dimensional workspaces may be created that are non-planer 
as discussed below. 

The planes and geometry shown in FIG. 11 may be 
provided as the default geometry for the graphical interface 

12. The system may provide the user with options to increase 
the depths and angles at which the basic geometry of the 
isometric workspace is defined. Preferably, the lines and 
shapes defining the boundaries of the three-dimensional 
workspace are not normally visible to the user, but during 
user customization, the lines demarcating the isometric 
space are made visible to the user. As noted above, the 
defined planes serve as a limit to the area in which a user 
may move a window on a particular plane or wall. For 
windows located in the perspective views or planes, the 
windows and text of the windows become smaller as the text 
approaches the line or point defined along the back edge of 
the plane. Generally, the point where objects appear smaller 
as the objects are positioned closer to the point is termed the 
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vanishing point. The vanishing point may be denned in 
memory for the display system. As illustrated, the window 
1110 may not extend or be placed above the upper edge 120 
of the right plane quadrilateral 84. Similarly, the window 

5 712 may not be positioned below the lower edge 122 of the 
right plane quadrilateral 84. The present invention may be 
implemented in a manner that reflects the depth of any object 
as a function of the objects location in the defined isometric 
room or three-dimensional workspace. 

10 It should be appreciated by those skilled in the art that the 
user viewpoint of the windows may be changed. Changing 
the user's viewpoint may be accomplished by changing the 
defined depth of the user relative to the windows (in the 
z-direction). Thus, moving the virtual location of the user 

15 from "in front" of the planes workspace, and moving the 
user into the workspace, allows the system to be used as an 
immersive interface with standard head tracking as used 
with a head mounted display. 
As known to those skilled in the art, a display screen may 

20 be divided into m by n elements generally known as pixels. 
A portion of the computer memory 20 is organized as a 
rectangular array of elements to represent the pixels of the 
display screen. The pixels as defined by the computer 
memory 20 may be addressed in terms of Cartesian coor- 

25 dinates (x, y) corresponding to each pixel. More specifically, 
for the planes, the computer system may store a set of 
Cartesian coordinates which identify the memory area for 
the planes. Those skilled in the art will appreciate that the 
boundaries defining the isometric room 80 can be readily 

30 established to provide the desired dimensions. 

As discussed above, the display screen 15 may be divided 
into m by n elements which are organized as rectangular 
array of elements in computer memory 20 and addressed in 
terms of (x, y) coordinates. Thus, lines or points defining the 

35 isometric room may be selected based on the (x, y) coordi- 
nates of the picture elements as represented in memory 20. 
When the desired points, lines, or shapes are selected for 
creating the planes, the location of the memory elements 
which correspond to the selected boundaries of the planes in 

40 (x, y) coordinates of the display buffer can be stored. The 
stored boundaries may be then referenced in execution of the 
steps of the present invention in order to provide defining or 
constraining regions for which a window located on a plane 
may be displayed. Well known methods of referencing a 

45 window being moved within the constraining boundaries of 
a display may be implemented to limit the movement of 
windows within the defined isometric space boundaries. The 
isometric room is defined as a set of perspective-transform 
objects. An isometric room is constructed by creating four 

50 objects of the class "perspective -transform". A first 
perspective-transform object's rotation is set to 90 degrees. 
The first perspective-transform object now has the transfor- 
mation information for the left perspective-transform. A 
second perspective-transform object's rotation is set to -90 

55 degrees. The second perspective-transform object also has it 
origin transformed so that it becomes the right perspective- 
transform. A third perspective-transform object has its origin 
set so that it becomes the back perspective-transform. The 
fourth perspective-transform object has no modifications to 

60 its origin or rotation and is thus the front perspective- 
transform. The size of each perspective-transform is a func- 
tion of the display resolution that is used on the display 
monitor. 

Windows are constrained to a perspective-transform by 
65 code in the window class that checks the coordinates of the 
window against the coordinates of the containing 
perspective-transform. If an attempt is made to move or size 
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a window beyond these boundaries, the window adjusts the 167. Referring to FIG. 14, a global view of the isometric 

coordinates to be within the boundaries of the containing space system 160 is shown in the upper right hand corner of 

perspective-transform. FIG. 14. The windows in the foreground of FIG. 14 is the 

The front and back perspective-transform share x and y isometric space 161. 

coordinates. The left and right perspective-transforms have 5 Cursor Operation 

a different x, y position for each perspective -transform. By As discussed above, the computer system of the preferred 

using different x, y positions for the left and right embodiment of the present invention provides depth cues to 

perspective-transforms, the user can position a window the windows. As also discussed above, a cursor is used to 

closer when on left perspective-transform than when it is on designate or select certain control items within the preferred 

the right perspective transform. This x, y coordinate infor- 10 embodiment of the present invention. To further enhance the 

mation is stored in separate member variables within the depth cues of the display system utilized with the present 

window class. This coordinate information can be within the invention, the computer system of the present invention 

perspective-transform itself and thus a window would not dynamically sizes the cursor according to the window on 

directly know where it is located in two-dimensional space. which the cursor is positioned and/or the position of the 

The window would only have information stored about its 15 cursor on a window. The cursor size reflects the depth of the 

width and height. As those skilled in the art will understand, location of the cursor in the defined isometric space. The 

the stacking or overlaying of windows in the three- preferred embodiment of the present invention considers a 

dimensional environment of the present invention is gener- location of the cursor to be on the surface of the plane 

ally handled in the same manner as a conventional window occupied by the window, whether the cursor is positioned in 

display environment. However, as noted herein, a window is 20 a normal, nonperspective transformed window, a perspec- 

placed, in the z-position, in the first available slot of a tive transformed window, or a zoomed out window, 

z-order list of the perspective- transform selected. With the Referring to FIG. 15, the dynamic cursor sizing operation 

present invention, windows on the front perspective- of the preferred embodiment of the present invention is 

transform 38 remain fully visible on the display screen when illustrated. The computer system of the present invention 

the window on the front perspective -transform 38 intersects 25 dynamically sizes the cursor 23 by automatically changing 

data from a window displayed on a side perspective trans- the size of the cursor 23 when the cursor 23 moves along a 

form. transformed window, "toward" or "away" from the user, 

Referring to FIG. 12, a non-planar bounding surface 141 when the cursor moves across a window border to another 

for defining three-dimensional depth perception is shown. window, or when the cursor moves from a window to a 

The bounding surface 141 is defined as a half sphere whose 30 non-window area of the display screen. The computer 16 

rim 143 meets the front bounding plane 147. Windows 151, (FIGS. 1 and 2) determines the location of the cursor 23 on 

153, and 157 are shown transformed to conform to the shape the display screen 15 as known to those skilled in the art, 

defined by the bounding surface 141. The windows 151, then resizes the cursor 23 to a predetermined size for the 

153, and 155 are transformed to their respective shapes with cursor 23 based on the particular location of the cursor on 

transformation matrices according to the window position 35 the display screen 15 and/or based on whether the cursor is 

defined by the bounding surface 141. Those skilled in the art positioned on a window at the cursor location. The cursor 23 

will appreciate that many other shapes of bounding surfaces is at its largest when it is not positioned on a window, 

may be defined for use as a bounding surface to provide Although multiple cursors 23 are illustrated in FIG. 15, 

three-dimensional graphics to a window based user inter- those skilled in the art should appreciate that generally only 

face. 40 one cursor is displayed at a time. The multiple cursors 23 

Multiple Three -Dimensional/Isometric Workspaces illustrated in FIG. 15 represent different positions 1510 of 

Referring to FIG. 13, another system for defining the the cursor 23 on the display screen 15. 

perspective-transforms based environment and for providing For example, the dynamic sizing of the cursor 23 may 

and defining the positions in which a window may be occur as follows. The cursor 23 at position 1510(a) is 

displayed is illustrated. A virtual isometric space system 160 45 considered to be "not positioned on a window". When a 

defines multiple isometric spaces 161, 163, 165, and 167. cursor 23 is moved from position 1510(a) to position 

Separating boundary planes 171, 173, 175, and 177 separate 1510(e) on the nontransformed window 1522, the computer 

the multiple isometric spaces 161, 163, 165, and 167. The system reduces the size of the cursor 23 at position 1510(e) 

separating boundary planes 171, 173, 175 and 177 intersect as compared to the cursor position 1510(a) which is not on 

a non-planar bounding surface 180. The non -planar bound- 50 a window. By reducing the size of the cursor 23, the 

ing surface 180 is cylindrical in shape. It should be appre- computer system indicates that the position of the cursor 23 

ciated that the non-planar bounding surface may vary in is further back in the display system. When the cursor 23 is 

shape. Each isometric space 161, 163, 165, and 167 may moved from position 1510(e) to either position 1510(b) or 

contain a set of planes and windows at various positions and 1510(c) on the zoomed back window 1532 the cursor 23 

configurations, consistent with their position in each iso- 55 becomes even smaller. When a cursor 23 is positioned on a 

metric space, as discussed above with the single isometric window or plane that is perpendicular to the user's view or 

space. A user can view a single space at a time such as the appears straight ahead in the user's line of view, the cursor 

single space 161. A user can access the space directly by size within the window that is perpendicular to the user's 

placing the cursor 23 in a selected isometric space and view remains the same. For example, the size of the cursor 

providing a selection single when the cursor is positioned in 60 23 located at position 1510(e) and the size of the cursor 23 

the selected space. Alternatively, the user can rotate to a at 1510(/) are the same. The size of the cursor 23 located at 

selected isometric space by selecting isometric space selec- positions 1510(a) and 1510(d) which are not positioned on 

tion controls 181 or 183. The isometric space selection a window or positioned in an open space are the same. The 

controls 181 and 183 operate to rotate through the isometric open space is considered to be perpendicular to the user's 

space as indicated by an arrow on the controls. It should be 65 line of view. Also, as indicated at positions 1510(6) and 

appreciated that animation effects may be displayed during 1510(c), the cursor's size is the same along the zoomed back 

transitions between the isometric spaces 161, 163, 165, and view of window 1532 Generally when the cursor is located 



12/23/2003, EAST Version: 1.4.1 



6,023,275 

21 22 

at the same z-position along a perspective transform or plane windows that are attached along the predefined boundaries 
as determined by the computer, the cursor size appears to be of the computer screen workspace. Perspective transforms 
the same on the display screen 15. apply any existing transformation information to the draw- 
As noted above, the size of the cursor 23 automatically i n g context before requesting all contained objects to draw 
changes when it crosses from and to objects located on 5 themselves into the drawing context. When the scene is 
different planes. By changing the size of the cursor 23 as it rendered, the group object notifies each perspective trans- 
passes across the border 1524 of window 1522 along the form in turn to render itself through the user of a draw 
path 1530, the cursor 23 appears to snap to, jump to or move Internallv this ^ w fiinction raakcs a set of caUs 
to a different plane Similarly, when the cursor 23 moves tQ ^ ^ cmKn{ transformation matrix to the dcsircd matrix . 

from position ISlmd) and across the border 1534 of window c i *u i a ~c * *u 

-iMi i *u *l 4«-n *l * *t_ 10 For example, the left perspective transform sets the matrix 

1532 along the path 1531, the cursor appears to snap to the iL * « j • j * * j ™ j 

window 1532 when it decreases in size as illustrated at so that dl drawing commands are rotated 90 degrees counter 

position 1510(c). The size of a cursor automatically snaps to clockwise so that a hne being drawn from eft to right will 

or resizes when the cursor 23 crosses the border of a window a PP car to BP awa y from the observer into the screen. After 

to another window or open space to indicate another depth the Perspective transform object has set the transformation 

level defined in the display system of the computer system. 15 matrix il then calls ^ c draw function for each of its 

Similar to the dynamic resizing of the cursor 23 as it contained objects. Normally these objects are window 
crosses the border of a window to a different window or to objects that only know how to draw in two dimensions, 
open space, the cursor 23 is dynamically resized as the Because the transformation matrix has rotated everything 
cursor 23 is moved along a perspective transformed window the window will appear to draw in three dimensions. Also 
in the z-direction. When the cursor 23 is moved on perspec- 20 the perspective transform object will adjust the transforma- 
tive transformed windows, such as window 1542, the size of tion matrix after each contained object has been drawn so 
the cursor 23 is dictated by the position of the cursor 23 and that the objects appear stacked on top of each other. Per- 
the location of the window. When the cursor 23 is moved spective transforms also utilize a z-order stacking as dis- 
"away" from the user such as along the z axis in FIG. 15, the cussed above to make sure that all of the contained windows 
size of the cursor 23 decreases. For example, the size of the 25 are drawn without overlapping visualizations, 
cursor 23 on the window 1542 at position 1510(/i) is larger As noted above, the operating system of the present 
than the size of the cursor 23 located at position 1510(t). invention uses windows extensively as part of the display 
Further, the size of cursor 23 located at position 1510(/) is system. Windows generally only have graphical information 
smaller than the sizes of the cursor 23 located at positions about their location in two-dimensional space (even though 
1510(i), 1510(g), and 1510(/i) on the window 1542. Thus, in 30 the underlying object class stores origin information for x, y, 
accordance with a method of the present invention the size and z coordinates) and some state information necessary for 
of the cursor 23 decreases in the direction away from the proper restoration from a perspective transformed position, 
user. A window may need to store state information about the last 

In addition to the resizing as discussed above, the speed perspective transform to which the window was attached so 
or animation of cursor transitions between windows at 35 that the window can be returned to the appropriate perspec- 
different depths may vary depending on the relative distance tive transform when the same perspective-transform button 
between transitions between windows. A cursor transition is selected twice. For example, if a window is currently on 
between windows can be considered an animated jump. The the front perspective transform, and the perspective- 
animated jump may be characteristically different when transform left button is selected, the window will be moved 
jumping up from a more distanced window in the three- 40 to the left perspective transform. The window stores an 
dimensional space than when jumping down from a window indication that it is leaving the front perspective transform so 
that is closer to the user. Animated jumps of the cursor may if the perspective transform left button is selected again the 
also have a characteristic sound associated with transitions window can be attached to the front perspective transform 
of various distances or depths in the three-dimensional again. The information about a window's two-dimensional 
space. 45 location could all be stored at the perspective transform level 
Object-Oriented Programming for Three-Dimensional Win- with a dictionary using the window object as the key and 
dows storing the two-dimensional location of the window on the 

As noted above, window display systems preferably are perspective transform as the value. When a window is 

implemented with object-oriented programming, Preferably, removed from either the front or back perspective transform, 

classes are implemented for all of the major functional 50 me perspective transform is saved so mat the window can be 

pieces of the interface. Application programming interfaces reattached when one of the other positioning controls is 

(API's) that provide three-dimensional transforms are pref- activated. The windows display themselves in response to 

erably used. The following description discusses the imple- requests from the owning perspective transform. Windows 

mentation of objects used with the present invention and also request the controls that are contained to draw them- 

how the different objects interact with each other. The 55 selves. 

following is a description of the classes that are utilized with A client area class is used to represent the client area of 

the present invention. a window. The client area is where an application performs 

With respect to the interface, the basic building block is its drawing. The API's are used to implement the drawing 
preferably a group. A group consists of a right and left and use display lists to optimize the drawing. Display lists 
perspective transform as well as a front and back perspective 60 provide a technique of preprocessing a sequence of drawing 
transform. As discussed herein, only one group is utilized, instructions and then referencing during the instructions 
but this is an implementation detail and not a limitation of through an integer identification. All visible components of 
the system. Multiple groups may be used. Groups have no the interface have an associated display list. In the case of 
visible portions. Groups only affect the location of the the controls (system menu, title bar, pushback, perspective- 
perspective transforms that make up the group. 65 transform left and perspective- transform right) a single 

Perspective transforms, as discussed herein, are generally display list is used that is a static member of the respective 

an invisible part of the interface that affect the display of class. All windows have a display list for their border and all 
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client area instances use a display list to store the drawing 
code necessary for displaying themselves. 

In order to optimize the drawing code when an object is 
in motion, a technique known as "scissoring" is used. This 
technique involves finding the minimum rectangular region 5 
of pixels that need to be refreshed and limiting the drawing 
to that set of pixels. Scissoring is a general name for a 
technique of clipping the drawing to a specific set of pixels. 
Scissoring is also known as marking. 

Display Processing Steps 10 

Hie preferred steps of the method for displaying or 
simulating a three-dimensional operating system are shown 
in FIGS. 16a and 16b. As noted above, information for 
displaying a window is received from a computer applica- 
tion and is displayed by manipulating conventional rectan- 15 
gular coordinate information to redraw the data on the 
computer screen using selected depth cues. The steps of the 
preferred method are implemented with a computer program 
operating on a general purpose computer such as the com- 
puter system 10 in FIG. 1. It will be appreciated that 20 
conventional software development techniques are used for 
developing the computer program implemented in conjunc- 
tion with the present invention. The preferred steps will be 
discussed in conjunction with FIGS. 1 and 2. 

Referring to FIGS. 1, 2, and 16a, the processes imple- 25 
mented in conjunction with the present invention are shown. 
At step 1602, the computer system 10 is powered up, and the 
operating system of the present invention is booted at step 
1604. At step i606, a computer user may invoke an 
application, or a routine specified during the system boot 30 
procedure may invoke an application. When an application 
is invoked, at step 1608, the window display system of the 
present invention displays the application data on the front 
perspective transform. At step 1612, the CPU 22 waits for a 
command from a computer user. When a command is 35 
received at the CPU 22, the process proceeds to step 1620, 
At step 1620, a determination is made as to whether the user 
command was a window transformation control or standard 
user command. If the command entered was a standard 
command, the display screen 15 is updated, at step 1626, by 40 
the CPU 22 according to the standard command. The 
process then proceeds to step 1612 to await another user 
command. If, however, at step 1612, the user selected a 
window transformation command, such as Push back 64, 
perspective-transform left 66, or perspective-transform right 45 
68, the process proceeds to step 1624 where the appropriate 
window transformation is implemented. 

Referring additionally to FIG. 166, the discussion of the 
preferred processes of the present invention is continued. At 
step 1702, the appropriate window data is accessed as 50 
discussed above. At step 1710, the CPU 22 determines 
whether the transformation command was a push back 
(zoom back) 64, perspective-transform left (PL) 66, or 
perspective-transform right (PR) 68 command. If the com- 
mand was a push back 64 command, the process proceeds to 55 
step 1714 where the appropriate zoom function is imple- 
mented as discussed above. If the command was a 
perspective-transform right 66 command, the process pro- 
ceeds to step 1712 where the perspective-transform right 
function is implemented as discussed above . If the command 60 
was a perspective-transform left 66 command, the process 
proceeds to step 1716 where the perspective- transform left 
function is implemented as discussed above. 

In summary, the preferred embodiment of the present 
invention provides a three-dimensional virtual workspace 65 
for window based display systems. The display system of 
the present invention is an isometric display system for an 
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operating system. The isometric display system provides a 
display with monocular depth cues by making automatic 
sizing and geometric transformations on two dimensional 
rectangles that define traditional windows. 

The isometric display system performs geometric trans- 
formation operations on rectangular windows to convey the 
impression to the user that the windows are positioned in a 
three-dimensional space. The impression of depth and of 
working in a three-dimensional space are created by trans- 
forming two-dimensional windows to appear as if the two- 
dimensional windows are embedded at orientations and 
positions in a three-dimensional isomeric space. The iso- 
metric display system responds to user activation of control 
buttons by transforming the shape and orientation of win- 
dows consistent with the defined three-dimensional space. 

The preferred embodiment of the present invention pro- 
vides left, right, front, and back planes or perspective 
transforms within the display system that operate to display 
an application or window on a user selected plane. The 
planes are preferably organized or defined as the walls of a 
four-sided room as seen when viewing the room from the 
outside. The planes defining the three-dimensional room or 
space are shaped to provide depth perception within the 
display monitor. The left and right planes narrow in dimen- 
sion as viewed from the front to back of the space defined 
on the display screen. Additionally, the front plane appears 
closer to the user than the back plane. The planes are 
preferably not visible but when a plane is selected by a user 
to display a window, the window's shape and contents are 
transformed or rescaled to conform to the shape of the 
planes. The simulated depth perception is created by trans- 
forming designated windows so that they conform to the 
dimensions of the respective plane or position on which the 
window is designated for display. The contents of displayed 
windows may be generally viewed in full when the windows 
are hung on different planes. 

The foregoing relates to the preferred embodiment of the 
present invention, and many changes may be made therein 
without departing from the scope of the invention as defined 
by the following claims. 

We claim: 

1. In a computer system operative to display computer 
application data in windows on a display monitor, said 
windows having borders defining an enclosed area for 
presenting said data on said display monitor, said computer 
system characterized by resources including a central pro- 
cessing unit (CPU), a memory, and display monitor said 
resources communicating via a system bus, said CPU opera- 
tive to control processes for defining the size and shape of 
displayed windows, a method for resizing a cursor on a 
display screen comprising: 

defining an isometric workspace on said display screen; 
determining the position of said cursor on said display 
screen; and 

resizing the cursor based upon the location of said cursor 
on said display screen to provide a three-dimensional 
transformed cursor to reflect depth in said isometric 
work space. 

2. The method of claim 1 wherein resizing said cursor 
occurs when said cursor crosses a border of a window. 

3. The method of claim 1 wherein resizing occurs based 
upon the cursor location on a window. 

4. The method of claim 1 wherein resizing causes said 
cursor to decrease in size as said cursor is positioned nearer 
to a selected position on said display monitor. 

5. The method of claim 1 wherein said computer system 
further comprises an operating system operative to define 
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said isometric work space on said computer monitor for 
providing depth cues to information displayed on said 
monitor and wherein the size of said cursor changes relative 
to the cursor location in said isometric work space thereby 
reflecting the depth of said cursor in said isometric work 5 
space. 

6. In a computer system operative to display computer 
application data in three-dimensional transformed windows 
on a display monitor, said three-dimensional transformed 
windows having borders defining an isometric space pre- 10 
senting said data on said display monitor, said computer 
system characterized by resources including a central pro- 
cessing unit (CPU), a memory, and display monitor said 
resources communicating via a system bus, said CPU opera- 
tive to control processes for denning the size and shape of 15 
said displayed three-dimensional transformed windows, a 
method for resizing a cursor on a display screen comprising: 

determining the position of said cursor on said display 
screen within the isometric space relative to the three- 
dimensional transformed windows; and 
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dynamically sizing the cursor based upon the location of 
said cursor on said display screen such that said cursor 
reflects a depth of the location of said cursor in the 
isometric space. 

7. The method of claim 6, further comprising: 
varying a speed of movement of said cursor depending on 

a relative distance between three-dimensional trans- 
formed windows to provide depth perception. 

8. The method of claim 6, further comprising: 
generating sounds dependent on relative movement of 

said cursor within the isometric space to provide depth 
perception. 

9. The method of claim 6, wherein dynamically sizing 
said cursor occurs when said cursor crosses a border of a 
three-dimensional transformed window. 

10. The method of claim 6, wherein dynamically sizing 
said cursor occurs base upon the cursor location on a 
three-dimensional transformed window. 
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